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ABSTRACT 


This essay adapts the concept of “shitty automation,” developed by Brian Merchant to name 
frustrating experiences with automated systems, to describe how human input—our labor, bodies, 
biases, prejudices, and desires—remains invisibly present in automated systems. Tracing a 
lineage of automation from Jacques de Vaucanson’s Canard Digérateur (1739) and Wolfgang 
von Kempelen’s mechanical Turk (1770) to contemporary artist Trevor Paglen, who uses 
Generative Adversarial Networks (GANSs) to create artworks, this essay considers how humans 
“stay in the loop” in automation and what “shitty automation” reveals about human culture, our 
desires, and the evolution of AI. 


INTRODUCTION 


In 1739, Jacques de Vaucanson, the French inventor already known for his extraordinary 
clockwork androids, debuted a new automaton. While his Flute Player (1738) could merely 
perform twelve melodies on a standard flute just as a human musician would, with skin-covered 
fingers and breath from hidden bellows, his new invention, Canard Digérateur (Digesting Duck) 
could do something more: it could shit.' [Figure 1] The life-sized, gold-plated creature was a 
marvel of naturalistic movement. It muddled water with its beak, reared on its legs, and quacked 
convincingly. Vaucanson claimed, and it was widely believed, that he had reproduced the duck in 
full anatomical detail, including its digestive system. The creature would eat from one’s hand and 
then evacuate its robotic bowels after the meal made its way through an internal “chemical 
laboratory.” The inventor did not claim to reap the benefits of the process, such as extracting 
energy, nourishment, or sustenance.” Vaucanson’s purpose here seemed to be the pursuit of a 
kind of unnecessary—even useless—labor for pure interest and delight.* Perhaps, then, 
Vaucanson’s art was in the duck’s gratuitous excrement as much as in its precision clockwork.* 


LE JOUEUR DE GALOUBET, LE CANARD ET LE JOUEUR DE TAMBOURIN 
PIÈCES AUTOMATIQUES CONSTRUITES PAR VAUCANSON. 


Figure 1. Henry René D’Allemange, Le Jouer de Gloubet, Le Canard et Le Jouer de 
Tambourin — Piéces Automatiques Construites par Vaucanson in Histoire des jouets 
(Paris, 1902), 222. Public Domain. 


Vaucanson’s art, however, was also in his artifice. It was some time before his Canard was 
exposed to be a “canard”: the “poop” was already hidden inside the duck by its human operator, 
awaiting the proper moment to be pushed out. Vaucanson’s careful reproduction of the duck’s 
anatomy, along with his written descriptions of the science of digestion and the physical and 
chemical processes hidden inside his machine, was all a sleight of hand meant to disguise the 
actual human hands and intervention necessary to complete the apparently autonomous and 
automatic process. The hoax begs the question, why would one even want robots to do such 
animal things as shit? Would not their freedom from biological needs and messes be a great 
advantage? And, even if authentic, why would we design our bots to do such useless, extraneous 
things as digest food unnecessary for their functioning? Vaucanson’s invention would have been 
impressive for its naturalistic movement alone without simulating digestion. Why, then, did the 
inventor feel the need to disguise this one bit of human aid in the process? In this essay I 
appropriate journalist Brian Merchant’s term “shitty automation” to bring to light the hidden 
human labor in automated processes, while also looking back to early automata to help us 
understand how such human (and animal) activities as defecating and making art came to be seen 
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as open to automation, and how this is reflected in contemporary experiments with art-making 
artificial intelligence bots. Merchant coins the phrase to describe commonplace and often 
frustrating experiences with automated systems that work so poorly as to require a human 
assistant to help the machine along with its task, such as grocery self-checkout lines and phone 
menus, which tend to make experiences worse for users without necessarily lowering consumer 
costs or wasted time (while eliminating paid jobs for human laborers).° Rather than focusing on 
systems that work so poorly as to need to be continually fixed by humans, I will adapt Merchant’s 
term to think more broadly about how humans “stay in the loop” in automation beyond being the 
necessary adjuncts for making the systems work. We see evidence of hidden human hands as the 
authors of the algorithms, curators of data sets, and ghost-work taggers of metadata that fuel the 
functions of AI bots and machine learning algorithms. Moreover, I want to think about the 
shittyness of automation not just as a result of its inefficiencies, frustrations, or failures, but as a 
way of describing the human elements—our labor, bodies, biases, prejudices, and desires—that 
remain invisibly present even when these systems are working perfectly or functioning “like 
magic.” In particular, I will look to how some of the nastier, abject bits of our culture are stowed 
inside these systems, placed there by human hands. In particular, I take up art-making Als (and 
their human collaborators) that seek to compete with people in their least “useful” but also most 
human of labors. If creative acts are often thought to be the only province of human endeavor safe 
from the automation revolution, what are we to make of the current generation of imaging bots 
trained on Generative Adversarial Networks (GANs), such as the one made by Trevor Paglen that 
“hallucinates” “photographs”? The disturbing images created by these systems, I argue, do not 
show the threat of automation to this last refuge of singularly human endeavor. Rather, the alien 
and abject qualities of the images are manifestations of the human hands that have shaped them, 
and are gestures that point to our overwhelming desire to see the aesthetic productions of Als fail. 


A REAL CANARD AND MERE BAGATELLE 


The renowned magician Jean-Eugéne Robert-Houdin was just one of the observers who 
uncovered Vaucanson’s dirty secret. A former clockmaker, he was fascinated by the 
technological skill needed to create this bit of artificial life. He went to see it in person at the 
Palais Royal in 1844 and was dutifully impressed.’ Sometime later, Robert-Houdin was asked to 
repair Canard Digérateur and uncovered its ruse. In his memoirs, he writes, “I found that the 
illustrious master had not disdained to have recourse to a trick which a conjuror would have been 
proud of. The digestion, so pompously announced in the memoir, was only a mystification—a 
real canard, in fact. Decidedly, Vaucanson was not only my master in mechanism, but I must bow 
before his genius for juggling.”* Vaucanson had employed as much magic as machine in this 
automaton. Even his description of the digestive process contained an expert deflection. Instead 
of detailing the process of the “chemical laboratory” purported to be in the duck’s guts, he made a 
provocative suggestion. This was intended to cause scientists and observers of the time to debate 
how digestion takes place in animals—whether by grinding or dissolution—rather than if his 
machine actually processed the food it consumed.” 
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Canard Digérateur was not the only animatronic hoax of its era. Better known and more 
confounding was Wolfgang von Kempelen’s chess-playing mechanical Turk (1770), which 
seemed to simulate human intelligence, not just animal biology. This device consisted of an 
exotically dressed android sitting at a chess board that rested on a large wooden cabinet filled 
with gears and levers. [Figure 2] Much to the dismay of its human challengers, who included 
Napoleon Bonaparte, Benjamin Franklin, and Charles Babbage, the Turk did not just play chess, 
he most often won.'° The Turk, like Vaucanson’s duck, was soon exposed to be a fraud, again one 
that aimed to disguise the human labor required for its marvelous performance. The automaton’s 
preternatural chess playing abilities were owed to a rather long series of chess masters who were 
hidden inside the machine’s base. [Figure 3] Ricky Jay, master of sleight of hand and scholar of 
the history of magic, described von Kempelen’s device as one that relied on the standard routines 
of magicians to allay the skepticism of viewers: 


To begin the demonstration, the doors and drawers of the cabinet were thrown open, 
preempting the audience’s suspicion that a human operator was concealed within the 
machines. Gears and wheels were displayed as a candle was passed behind the dimmest 
parts of the interior. The Turk’s robes were unceremoniously thrown over his head to 
allow for inspection from the rear, and even the interior of his physique was disclosed 
through a small door opened beneath his trousers. The entire structure was mounted on 
casters so that the cabinet could be revolved and viewed from all angles.'! 
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Figure 2. Joseph Racknitz, illustration of von Kempelen’s Turk from Ueber den 
Schachspieler des Herrn von Kempelen und Dessen Nachbildung (Leipzig and Dresden, 
1789), 63. Public Domain. 
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Just as a magician might allow the audience to peek inside a hat before pulling out a rabbit, or 
inspect the casket before sawing a woman in half, von Kempelen (and those who toured the Turk 
after him)!’ made a spectacle of revealing the complex and apparently uninhabited interior. As he 
rotated the box, the hidden player shuttled himself into secret compartments, following a tight 
choreography that kept him continually hidden from view. During the match, von Kempelen used 
other tricks to divert the audience’s attention. According to Jay, he busied himself with a separate, 
small wooden box, which he carefully guarded, not allowing anyone to see inside. The box had 
no purpose, but this bit of classic misdirection helped distract from the shuffling noises and lamp 


smoke coming from the base.’ 
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Figure 3. Joseph Racknitz, illustration of von Kempelen ’s Turk from Ueber den 
Schachspieler des Herrn von Kempelen und Dessen Nachbildung 
(Leipzig and Dresden, 1789), 65. Public Domain. 


Despite the sensation and wonder it caused, von Kempelen derogatorily described his own 
invention as “mere bagatelle’—it was a trifle, a simple game, an act of little value or importance. 
What contemporary commenters may have taken as modesty or dissimulation was, perhaps, just 
simple honesty. To admit that Vaucanson’s duck was a “canard” and von Kempelen’s Turk “mere 
bagatelle,” however, is not to disregard their technical and conceptual importance or to downplay 
the impact they and their inventors would have on the future of automation and labor. Jessica Riskin 
describes Vaucanson’s inventions—authentic and fraudulent alike—as “philosophical 
experiments” that attempt “to discern which aspects of living creatures could be reproduced in 
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machinery, and to what degree, and what such reproductions might reveal about their natural 
subjects.”'* The importance of chess playing and defecation, in the eighteenth century at least, was 
that they, according to Riskin, seemed to be outside what it seemed possible to mechanize and 
thereby “provoked mechanicians who were interested in testing the limits of their crafts to become 
conjurers.”'> These early ventures into artificial life and artificial intelligence seem to be perfect 
tests for thinking through “Clarke’s Three Laws,” formulated by science fiction author and futurist 
Arthur C. Clarke: 


“When a distinguished but elderly scientist states that something is possible he is almost 
certainly right. When he states something is impossible, he is very probably wrong.” 
“The only way of discovering the limits of the possible is to venture a little way past them 
into the impossible.” 

“Any sufficiently advanced technology is indistinguishable from magic.”"° 

It may have seemed, and still might, that automating intelligence, creativity, gross anatomy, or 
biological processes would be impossible. These two automata could appear to prove it with their 
failures, but at the same time, by means of subterfuge and guile, they also blurred the boundary line 
between animal and machine by taking the perilous step into the impossible. They were able to do 
so because of Clarke’s third law: just like legitimate technological advancements, Canard 
Digérateur and the Turk were so miraculous that they were indistinguishable from magic, or in this 
case, they were magic. 


The inventions of Vaucanson and von Kempelen, two mechanics-turned-magicians, had great 
impacts on labor as much as on philosophy. The Turk has famously become a metaphor for “ghost 
work” in the current gig economy. In 2005, Amazon.com launched an online labor market which 
took its name from von Kempelen’s automaton and, like the original, sought out human labor to 
secretly power its machines. Amazon’s Mechanical Turk, or MTurk, for short, pays distributed 
workers to perform small tasks. This began as cleaning up copy on Amazon’s website and soon 
expanded to an API (and spawned competitors) used by numerous companies to staff a wide 
assortment of tasks, including many with the express purpose of helping automated systems and 
Als with little jobs that they cannot complete by the power of their own “intelligence.”'’ These 
labors, too, are “mere bagatelle’—typically trivial, “micro-tasks” for little compensation. Mary L. 
Gray and Siddarth Suri call the intentionally hidden human labor that powers apps, websites, and 
artificial intelligence systems “ghost work.”'* Ghost work, in turn, helps fill in these gaps and 
simultaneously improves the AI’s future performance by training the systems on the skills the 
workers are currently augmenting. This labor might take the form of tagging objects in images or 
cleaning up machine translated text. With each task they complete, then, the ghost laborers work 
toward eliminating the need for their help with that chore. Gray and Suri do not see this process, 
however, as a risk to the demand for future ghost work. Once ghost work helps perfect a given AI 
skill, “workers move on the next task engineers assign them that push the boundaries of 
automation,” which then creates a new gap.'’ We can expect, then, that humans will always be 
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needed somewhere along the line to assist with even the most advanced automated tasks. In 
theory—and likely in practice—there will be no end to shitty automation. 


Vaucanson’s duck, too, led to revolutions in industrial labor and redefined what acts were 
considered to be open to automation and mechanization. Vaucanson’s various automata, though 
failing to produce either shit or music, were far from useless, as they helped him reconceptualize 
what was intelligent labor and which tasks required human creativity. King Louis XV was so 
impressed with Vaucanson’s abilities that he appointed the inventor “Inspector of Silk 
Manufacture.” During his tenure, Vaucanson invented an early automatic, programmable loom 
(1747, later to be improved by Jacquard), which, Riskin argues, “took over a function that had 
hitherto been, not only human, but highly skilled.” Canard Digérateur led to a process of opening 
the arts up to automation, and seeing these hitherto animal or human processes as programmable. 
The Turk, on the other hand, created an illusion of automation and accepted (although also 
disguised) the fact that some human labor will be required, no matter how autonomous actions 
might appear. Within these two objects and the histories they catalyzed we can see the origins of 
our current logic of automation and intelligence: all fields, all practices, regardless of how 
specifically human they might seem to be, will be open to automation, particularly via AI, and the 
human labor needed for those apparently autonomous systems will be hidden all along the way. 


Merchant’s examples of shitty automation involve poorly functioning, inefficient technology and 
the very visible and acknowledged presence of a human stepping in to help a system complete its 
task. Looking back to Vaucanson’s and von Kempelen’s automata, we can see another lineage for 
shitty automation: invisible, often unrecognized human labor makes systems function “like magic.” 
“Working like magic,” however, is a dangerous illusion, especially where automation is concerned. 
As M.C. Elish of Data & Society writes, this common “refrain from the marketing materials of new 
technologies, especially those involving AI, ... implies impressive and seamless functionality and 
the means by which the effect was achieved is hidden from view or irrelevant. Yet, from another 
perspective, implying something works like magic focuses attention on the end result, denying an 
accounting of the means by which that end result was reached.””' This is troubling, Elish explains, 
because “AI seems to promise not only efficiency and insight, but neutrality and fairness,” the latter 
being qualities often thought impossible to achieve with human input because of our implicit and 
explicit biases.” But as we have already seen with the example of MTurk, humans are often 
involved somewhere on the back end. As in Merchant’s examples, humans are involved, and their 
labors add to the system’s efficiency. What makes the systems “shitty,” we shall see, are the ways 
in which cultural biases enter AI algorithms and data sets via these human factors, and how the lack 
of transparency and accountability in such systems, especially as they are “increasingly positioned 
as the answer to every question,” often exacerbate the structural inequalities for vulnerable 
populations while masquerading as objective and rational.” 


HUMANS IN THE LOOP 


In computer science, augmentation and artificial intelligence initially seemed to be two distinct 
research fields. At the dawn of the computer age in the late 1960s, the two pursuits were “based on 
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fundamentally different philosophies”: one “sought to augment the human mind; and the other 
replace it.” When Douglas Engelbart demoed his oNLine System (NLS) (an event commonly 
known as “The Mother of All Demos”) in 1968, which debuted new-fangled technologies and 
concepts such as a dedicated personal computer with a monitor for graphics and text editing, 
hyperlinked documents, an interactive computer “mouse,” and the beginnings of an interconnected 
network of distant computers (ARPAnet, which soon became part of the technical basis of the 
Internet), it seemed that augmentation had, at least momentarily, won out over artificial 
intelligence.” The exciting future of computing that Engelbart presented, John Markoff explains, 
caused the tables to turn: “computer science, which had until then been primarily concerned with 
the esoteric problem of automating human intelligence, would never be the same.” Most notably, 
the trends Engelbart began would keep humans “in the loop” in the future of computing, an impulse 
antithetical to the general goals of AI research of the era. “Engelbart,” Markoff writes, “was a 
heretic, and it was from his heresy that personal computing grew.”’ Counterintuitively, perhaps, 
the AI boom—in both its power and ubiquity—could not take place without the technologies of 
augmentation Engelbart pioneered or without the copious amounts of data the humans using them 
would produce. 


It is easy to see how the heretical technological future Engelbart outlined in his “Mother of All 
Demos” became the basis for the ghost-work-fueled AI revolution of today: web-connected 
workers stare at their screens, hold their mice, tag images, answer questions, link and upload 
documents that train AI. “Humans in the Loop,” or HITL, is now a standard method in AI 
research to create machine learning models in which humans train, fine tune, and test algorithms. 
Even when systems operate fully autonomously, there were often humans in the loop somewhere 
earlier on in its development, as described above. It is this hidden human labor that helps our apps 
and automated systems “work like magic.” But it is not just ghost work via Engelbart’s 
augmentation tools that has aided rapid advancement of AI.”* The astounding improvements in AI 
in recent years are the result of in-machine learning, which could not succeed without the massive 
amounts of data present on the Internet, much of it uploaded, tagged, and defined by people in 
their leisure time “for free.” “The personal computer,” Ethem Alpaydin writes, “was the first 
step in making computers accessible to the masses; it made digital technology a larger part of our 
lives and most important for [machine learning], allowed more of our lives to be recorded 
digitally.” So began the process of converting our lives to data; sharing that information over the 
web, then turning it into training sets for machine learning algorithms. Until recently, before 
machine learning, AI was trained by programmers who defined what a bot knew and what it had 
to do. But the glut of information—audio, visual, and textual—produced, processed, stored, and 
shared online led to the development of learning programs that can produce their own models 
from ever-expanding data sets and can creatively generate new information and predictions from 
what they learn. 
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Figure 4. Trevor Paglen, The Shark, Linear Classifier (2018). Dye sublimation print, 
48 x 48 inches. Edition of 5. Courtesy of the artist, Metro Pictures, New York and Altman 
Siegel, San Francisco. 


Artist Trevor Paglen, whose work has continually sought to expose “invisible” elements of our 
military-industrial world—from secret bases to drones, undersea cables, and spy satellites—has 
recently begun figuring the “invisible images” that Als make and show to each other in the 
processes of machine learning. “Human visual culture,” Paglen explains, now comprises only a 
very small subset of “images” in circulation today. Many images, perhaps the vast majority of them 
“are now made by machines for other machines, with humans rarely in the loop.”*! Working with 
computer scientists at Stanford University, Paglen developed software that could intervene in the 
system to produce a human-readable, visual image of the machine learning data. Abstracted from 
the compilation of thousands of reference images in the popular ImageNet training set, Paglen’s 
“photographs” manifest visions of how these linear classifiers understand the visual world.” The 
resulting images are something like the AI’s version of a Platonic ideal. Shark, Linear Classifier 
(2018) [Figure 4], for example, pictures the eponymous shark as hazy white-yellow glow hovering 
in the center of a murky blue-green expanse. Once so named, it is not hard to imagine how this 
generalized image of a shark derived from the sample set. With all of the vast variety of the more 
than 500 species of sharks—from hammerheads to tiger sharks, great whites, and enormous whale 
sharks—the idea of shark becomes, like the picture, very fuzzy. In Merchant’s sense, this is a 
“shitty” image, in that it seems to fail informatically for the viewer and needs to be helped along 
by the title and the institutional exegesis in exhibition wall texts and artist statements. Without 
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them, the picture is a lush but potentially unremarkable and meaningless abstraction. Labeled as 
such, however, the AI’s soft abstraction becomes menacing: it is the flashing white belly of a 
predator spotted in one’s peripheral vision, emerging from an empty and inhuman environment. 
Any failure to signify or cohere is not a flaw but Paglen’s intentional solicitation of a technological 
sublime.” It is an illustration of an othered nature, of a parallel world that is not “for us,” that 
exceeds and disregards our ability to understand. Paglen’s software and its “photographic” output 
allow one to see—to conjure from code—what this AI imagines a shark to be. No matter how 
unnerving one might take this gesture toward an inhuman technological singularity to be, the real 
threat is the very human biases and flaws that inform its evolved and automated vision. 


Paglen’s process of translating the data generated by the linear classifier into an image is an act of 
inserting a human into the loop of a system that otherwise would seem to be operating 
autonomously. The machine learning algorithm teaches itself based upon the available data. The 
image that results here is an intervention that allows for a moment of aesthetic judgement and 
philosophical contemplation. But the assertion that, up until this moment of mediation, there were 
no humans in the loop would have to be based on a very narrow view of the process. As in the 
Turk, human labor is invisibly hidden all along the way, and it sneaks bits of our human “shit”— 
our tastes, our assumptions, our conscious and unconscious prejudices—into the system. All of 
the over fourteen million images on ImageNet use crowdsourced human labor from Amazon 
MTurk for “hand” annotation and quality control.** Already in 2012, The New York Times 
reported that each year ImageNet “employs 20,000 to 30,000 people who are automatically 
presented with images to label, receiving a tiny payment for each one.”** There is no true 
elimination of the human hand in these machine learning algorithms, despite their automation. 
While we might imagine that AI is objective and unbiased, operating above the fray of petty 
human irrationalities and cultural baggage, the opposite is true: machines may be doing the 
learning, but humans dictate what they can see. Human bias enters into the system when workers 
tag the images and curate the datasets, as well as when software applications are developed. As 
demonstrated in recent PR debacles, image recognition software might think that any person in a 
picture of a kitchen must be a woman if its image set skews that way,” or that every black human 
face is actually a gorilla, if its image set isn’t representative of the diversity of human skin 
tones.” Neural networks, Paglen explains, “cannot invent their own classes; they’re only able to 
relate images they ingest to images that they’ve been trained on. And their training sets reveal the 
historical, geographical, racial, and socio-economic positions of their trainers.... Formal concepts 
contain epistemological assumptions, which in turn have ethical consequences.”** But AI does 
not just reproduce the biased information input, it learns to essentialize from it. Machine 
intelligence, Mateo Pasquinelli writes, “is not biomorphic—it will never be autonomous from 
humankind.... Machine intelligence is sociomorphic, but not in a good way....We feed 
algorithms our racist, sexist, and classist biases and in turn they warp them further. As Marx 
knew, absent political action, machines do not just replace but amplify previous divisions of labor 
and social relations.”*’ That is, when facial recognition algorithms tag dark-skinned faces as apes 
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or when recruiting software automatically rejects applications by women,” these are not glitches; 
these are clear manifestations of the humans in the machine. 


MONSTERS 


In another series of AI works Paglen actively seeks out and exploits the human traces in data sets 
and machine learning algorithms. Rather than producing a work that exposes what an AI might 
understand a shark, goldfish, sun, or stealth bomber to be, his Adversarially Evolved Hallucination 
series uses Generative Adversarial Networks (GANs) to categorize images and to imagine new 
ones. While this could be done with classes such as “shark,” Paglen chooses far more idiosyncratic 
types to expose just how pervasively human culture affects the outcomes of these apparently 
independent and autonomous machines. 


GANs employ two Als that compete with one another. The first, the “discriminator” is trained to 
identify and class a certain type of image, as in the shark example above. A second system, the 
“generator” attempts to “‘evolve’ recognizable images from random noise—almost like a 
hallucination” that the discriminator could accept as members of that class.*' Paglen produced his 
training sets, or “corpuses,” to identify less stable categories of objects, such as metaphors for the 
horrors of capitalism (“Monsters of Capitalism”) or symbolic items that appear in Sigmund Freud’s 
Interpretation of Dreams. Rather than simply exposing AI as a manifestation of the unconscious 
biases of all of the various humans in the loop, Paglen intentionally curses his bots with neuroses 
or imbues them with poetic imaginations and tendencies toward metaphor. Vampire (Corpus: 
Monsters of Capitalism) (2017) [Figure 5] is an example of a picture that passed his discriminator’s 
test for a likeness of the imaginary monster from folklore, literature, and film. The image, as a 
whole, looks little like Bella Lugosi’s or Christopher Lee’s Dracula or Al Lewis’s Grandpa 
Munster: there are no fangs or widow’s peaks, Victorian cravats, or rubied amulets. It looks barely 
human(oid) at all; but all of the essential elements are there, even if they are a bit out of order: a 
dark dilated pupil of a human eye appears through a hole in what might be a cascading curtain of 
silk or velvet; a bright white patch reminiscent of pancake makeup defines a potential cheekbone, 
and a viscous red and rusty orange spill slides down what could be a chin from the gaping ledge of 
an open maw; a bright silvery shine dominates the top of the picture, giving the effect of moonlight 
bouncing off pale skin or a slick swath of Brylcreem-ed hair. The result is not a rational 
“intelligence,” but a flawed and nervous character that seems paranoid, seeing monsters in shadows 
or in just the barest indications of faces. 
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Figure 5. Trevor Paglen, Vampire (Corpus: Monsters of Capitalism), Adversarially 
Evolved Hallucination (2017). Dye sublimation print, 60 x 48 inches. Edition of 5. Courtesy of 
the artist, Metro Pictures, New York and Altman Siegel, San Francisco. 
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Figure 6. Trevor Paglen, False Teeth (Corpus: Interpretations of Dreams), 
Adversarially Evolved Hallucination (2017). 21 % x 26 % in. Edition of 5. Courtesy of the 
Artist and Altman Siegel, San Francisco. 


Paglen pursues this line of intentionally programming for irrationality and neurosis by developing 
a GAN that parses and produces images based in Sigmund Freud’s Interpretation of Dreams. 
Rather than looking at images and identifying specific individuals or the products and objects 
pictured, as many visual recognition systems that comb through our social media posts do, Paglen’s 
sees only the items Freud marked as symbolically significant in his landmark text: it sees 
“injections, puffy faces, scabby throats, windows, ballrooms, ... mothers.’“? False Teeth (Corpus: 
Interpretations of Dreams) (2017) [Figure 6], for example, presents the viewer with a painterly 
dreamscape that appears to owe some of its sensibility and palette to Philip Guston or Jean Fautrier. 
Mottled areas of white, pink and red softly collide at dark edges. There is a sense of flesh, if not 
figure, but only as half-remembered, repressed, or dreamed. A row of rough-hewn, blocky white 
and silver teeth rest in the center of the frame, eerily tucked into a small pocket of shadow. It seems 
impossible to imagine this image being useful in any way other than to mystify, communicate, 
interest, or delight. Paglen’s photograph appears to be art made by one AI for another: the generator 
exposing the oddity of its secret flawed soul and the discriminator finding it relevant and 
meaningful. By embracing this very particularly human psychoanalytic subjectivity, and 
foregrounding his own presence and interventions in the process—from tagging the training set to 
choosing which of the generated images to print and display—Paglen develops Als that are “not 
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useful in terms of capitalism or policing, the normal kinds of things you use artificial intelligence 
for.” The images produced by Paglen’s nervous bots, like those from his linear classifier series, 
also fail to cohere. As useless as these images might be, the twisted visions they produce are 
pleasing on yet another level—they seem to confirm and reinforce (if only by Paglen’s own 
selection) the very human desire to remain superior and in control, which is particularly appealing 
in the era of “deep fakes” and other entirely convincing simulations enabled by machine learning 
and AI. To see these bots’ confused productions, their inaccurate renderings and flawed 
imaginations, is to fulfill our own unspoken desires: if Als are going to make art, let it be shitty. 


Frankenstein may not have been one of Paglen’s “monsters of capitalism,” but certainly he has 
long been an apt reference when discussing automatons, automation, and AI. Mary Shelley 
subtitled her 1816 book “The Modern Prometheus” after the Titan who created man and gave 
them fire, an insult to the gods who then released Pandora and all her curses as punishment. 
Historians suspect that Shelley saw 18"-century automata, such as Pierre Jaquet-Droz’s writing 
automaton soon before beginning Frankenstein.“ The name “Frankenstein” seems to cause 
continual confusion, as typically it is used to mistakenly refer to the “creature” in Shelley’s book 
rather than to its creator, Dr. Frankenstein. Any quick search for a Halloween costume will 
underscore this tendency. Rossiter Johnson writes in his 1908 introduction to the story, “It is 
strange to note how well-nigh universally the term ‘Frankenstein’ is misused, even by intelligent 
persons, as describing some hideous monster; whereas Frankenstein is the name of the hero and 
supposed narrator of the story and not his terrible creation.” This “hero,” however, gave birth to 
a being and then disowned and abandoned it; he refused to take responsibility for its actions or his 
role in them, or to teach it better ways. If one calls “the creature” Frankenstein, it is a proper 
assignment of parentage and influence. Perhaps it is right that both names should be flipped: the 
creature is Frankenstein and the doctor is, rightly, “the monster.” Only in this apparent misnomer 
do we see clearly who was truly monstrous. Paglen, on the other hand, takes ownership of these 
monstrous images: they are created by a process that he put into motion. It is not a ceding of 
authorship or authority to a bot, but rather, Paglen explains, it is like the conceptual processes of 
Sol Lewitt, in which one is “just using a very different set of tools to make [art].... You’re setting 
up elaborate sets of rules, and the images you’re putting out are essentially the output of that 
system.”*° Riskin, above, described the importance of Vaucanson’s and von Kempelen’s 
fraudulent automata as intellectual experiments aimed “to discern which aspects of living 
creatures could be reproduced in machinery, and to what degree, and what such reproductions 
might reveal about their natural subjects.” Paglen’s bots, too, indicate that one should focus 
attention not on the algorithms, but on the humans who had invisible hands in their creation. 
What one sees in the images created by Als, then—Paglen’s as well as those in our broader 
culture—is not an alien world but a dark mirror held up to our own. 
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